Parametric Hand Tracking and Its Applications

An approach for estimating the 3D absolute position and orientation of the hand in space using parametric modelling of the central region of the hand and some of its applications are presented. A stereo camera is used to first build a preliminary disparity map of the hand. Then, the best fitting plane to the disparity points is computed using robust estimation. Next, the 3D hand plane is calculated based on the disparity plane and the six parameters of position and orientation of the hand are estimated. Tracking the hand region over a sequence of frames and coping with noise using robust modelling of the hand motion enables us to estimate the trajectory of the hand in space. Experimental results demonstrate the accuracy of this technique. A virtual drawing system and a virtual marble game are presented as two sample applications. In virtual drawing, the 3D position of the hand in space is estimated. Then, by tracking the central region of the hand in 3D and estimating a virtual plane in space, the intended drawing of the user is recognized. This system can be used to communicate drawings and alphabets to a computer where a classifier can transform the drawn alphabets into interpretable characters. It is also shown how this system can be used to create multi-stroke drawings. In the virtual marble game, instantaneous orientation of the hand is simulated to render a graphical scene of the game board. Real-time visual feedback allows the user to navigate a virtual ball in a maze.